* • 



L Number 


Hits 


Search Text 


DB 


Time stamp 


1 


176 


scene$3 with (subdivid$6 divid$6) with 


US PAT 


2003/11/21 


14: 


55 






frame $5 










2 


89 


scene$3 near8 (subdivid$6 divid$6) near8 


US PAT 


2003/11/21 


15: 


09 






frame $5 










3 


3626 


frame near9 start$6 near9 end$5 


US PAT 


2003/11/21 


15: 


12 


7 


630 


(frame$3 near4 point$3) with ( (start$5 


US PAT 


2003/11/21 


16: 


04 






begin$8) and end$3) 










8 


30 


(frame$3 near4 indicat$6 near5 point$3) 


US PAT 


2003/11/21 


16: 


05 






with ( (start$5 begin$8) and end$3) 










9 


1 


6353461. pn. and (audio voice$3) 


US PAT 


2003/11/21 


16: 


26 


10 


1 


6353461. pn. and {comment$5) 


US PAT 


2003/11/21 


19: 


06 


11 


1 


6353461. pn. and (network$3) 


US PAT 


2003/11/21 


16: 


41 


12 


1 


6353461. pn. and (database$5) 


US PAT 


2003/11/21 


16: 


53 


13 


1 


6353461. pn. and (file$3 generat$6) 


US PAT 


2003/11/21 


17: 


53 


14 


158 


(file$3 adj generator) 


USPAT 


2003/11/21 


18: 


14 


15 


11 


(file$3 adj generator) with database$5 


US PAT 


2003/11/21 


17: 


34 


16 


0 


6353461. pn. and (filter$5) 


USPAT 


2003/11/21 


16: 


56 


17 


11 


(file$3 near generator$3) with database$5 


USPAT . 


2003/11/21 


17: 


38 


18 


222 


(file$3 near generat$5) with database$5 


USPAT 


2003/11/21 


17: 


38 


19 


1 


6353461. pn. and {file$3 ) 


USPAT 


2003/11/21 


17: 


53 


20 


1 


6353461. pn. and (folder$5 database$5) 


USPAT 


2003/11/21 


18: 


43 


21 


1 


6353461. pn. and (folder$5 director$5 


USPAT 


2003/11/21 


17: 


56 






database$5) 










22 


1 


6353461. pn. and (folder$5 directory 


USPAT 


2003/11/21 


18: 


13 






directories database$5) 










23 


38 


{file$3 adj generator) with {retriev$6 


USPAT 


2003/11/21 


17: 


57 






stor$6) 










24 


4 


(file$3 adj generator) with {directories 


USPAT 


2003/11/21 


18: 


02 






directory) 










25 


9 


(file$3 adj generator$3) with (media 


USPAT 


2003/11/21 


18: 


02 






multimedia audio video) 










26 


1 


6353461. pn. and {'58b') 


USPAT 


2003/11/21 


18: 


13 


27 


15 


(file$3 adj generator$5) with (delet$5 


USPAT 


2003/11/21 


18: 


14 






add$3 ) 










30 


6 


(file adj generator$5) with edit$3 


USPAT 


2003/11/21 


18: 


31 


31 


7 


( (media multimedia file) adj generator$5) 


USPAT 


2003/11/21 


18: 


32 






with edit$3 










32 . ..... 


50 


( (media multimedia file) adj generator$5) 


USPAT 


2003/11/21 


18: 


40 






with (generate generating edit$3 ) 










33 


296 


generated near media 


USPAT 


2003/11/21 


18: 


40 


34 


0 


(generated near media) with reduce with 


USPAT 


2003/11/21 


18: 


41 






(buffer$3 memor$5) 










35 


4 


(generated near media) with (buffer$3 


USPAT . 


2003/11/21 


18: 


41 






memor$5) 










36 


1 


6353461. pn. and (rate$5) 


USPAT 


2003/11/21, 


18: 


58 


37 


4 


6353461. uref . 


USPAT 


2003/11/21 


19: 


03 


38 


0 


63534 61 . uref. and comment$3 


USPAT 


2003/11/21 


19: 


04 


39 


6 


"6353461" 


USPAT; 


2003/11/21 


19: 


03 








US-PGPUB; 














EPO; JPO; 














DERWENT ; 














IBM TDB 








40 


8 


(("6154600") or ("6094522") or ("6092119") 


USPAT 


2003/11/21 


19: 


04 






or ("6091778") or ("6085019") or 














("6067126") or ("6064793") or 














("6044198") ) .PN. 










41 


0 


((("6154600") or ("6094522") or 


USPAT 


2003/11/21 


19: 


04 






("6092119") or ("6091778") or ("6085019") 














or ("6067126") or ("6064793") or 














("6044198") ). PN. ) and comment $3 










42 


1 


6353461. pn. and ( (user$3 editor$5 


USPAT 


2003/11/21 


19: 


07 






director$3) with comment$5) 










43 


1 


6353461. pn. and (operator$5 director$5 


USPAT 


2003/11/21 


19: 


22 






editor$5) 










44 


1 


6353461. pn. and (name$3 list$3) 


USPAT 


2003/11/21 


19: 


12 


45 


17 


(script$5 editor$5) with list$3 with 


USPAT 


2003/11/21 


19: 


17 






comment $3 










46 


35 


( edit$5) with list$3 with comment$3 


USPAT 


2003/11/21 


19: 


17 


47 


1 


6353461. pn. and (software$5 program$5 


USPAT 


2003/11/21 


19: 


29 






instruction$3) 
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48 


1 


6353461. pn. and (disk$3 memor$3 buffer$5 


US PAT 


2003/11/21 


19 


34 


49 


1 


cd) 

6353461. pn. and (review$5 ) 


US PAT 


2003/11/21 


19 


35 


50 


1 


6353461. pn. and (review$5 with database$5 


US PAT 


2003/11/21 


19 


36 


51 


0 


) 

63534 61. pn. and (review$5 with (dynamic$5 


US PAT 


2003/11/21 


19 


41 






automatic$6) ) 










52 


1 


63534 61. pn. and ( dynamic$5 automatic$6) 


US PAT 


2003/11/21 


19 


37 


•J -> 


1 


& S "\ 4 f\ 1 Tin anH / rpvi pwSS wi t* h 


US PAT 


2003/11/21 


19 


41 






(instruct$5) ) 










54 


1 


6353461. pn. and (review$5 ) 


US PAT 


2003/11/21 


19 


42 


55 


1 


6353461. pn. and (review$5 with system$3 ) 


US PAT 


2003/11/21 


19 


43 


56 


1 


6353461. pn. and (review$5 with playback$5 


US PAT 


2003/11/21 


19 


47 


57 


1 


) 

6353461. pn. and (comment$5 with edit$6 ) 


US PAT 


2003/11/21 


19 


47 


- 


10 


editor$5 with review$6 with comment$3 


US PAT 


2003/11/20 


14 


42 


- 


0 


6262724. pn. and (review$5 approv$6 


US PAT 


2003/11/19 


13 


28 






comment$5) 










- 


1 


6262724. pn. and (edit$6) 


US PAT 


2003/11/19 


13 


28 


- 


1 


(editor$5 with review$6 with comment$3) 


US PAT 


2003/11/19 


14 


54 






and (player$5 ) 










- 


250 


editor$5 with comment$3 with edit$9 


US PAT 


2003/11/19 


14 


54 


- 


2 


editor$5 with comment$3 with edit$9 with 


US PAT 


2003/11/19 


15 


40 






(approv$5 reject$5) 










- 


0 


ladp with director$3 with gateway$3 


US PAT 


2003/11/19 


15 


41 


- 


22 


ldap with director$3 with gateway$3 


US PAT 


2003/11/19 


15 


50 


- 


1 


6353461. pn. and (player$3) 


US PAT 


2003/11/19 


15 


54 


- 


1 


6353461. pn. and (window$3) 


US PAT 


2003/11/19 


15 


58 


- 


1 


6353461. pn. and (approv$5 reject$5 


US PAT 


2003/11/20 


14 


42 






review$6) 










- 


1 


6353461. pn. and (editor$5 reviewer$5) 


US PAT 


2003/11/19 


16 


00 


- 


1 


6353461. pn. and ( (edit$5 review$5) with 


US PAT 


2003/11/21 


16 


22 






comment$5) 










- 


10 


(("5978648") or ("5999173") or ("6426778") 


US PAT 


2003/11/21 


19 


15 






or ("6489969") or ("5995951") or 














("6201924") or ("6212527") or ("5852435") 














or ("6484199") or ("6411725") ) . PN . 










- 


2 


((("5978648") or ("5999173") or 


US PAT 


2003/11/19 


16 


07 






("6426778") or ("6489969") or ("5995951") 














or ("6201924") or ("6212527") or 














("5852435") or ("6484199") or 














("6411725") ) .PN. ) and ( (edit$5 review$5) 














with comment$5) 










- 


6 


(("5467288") or ("5724605") or ("5752029") 


US PAT 


2003/11/19 


16 


14 






or ("5754851") or ("5946445") or 














("6058236") ) .PN. 










- 


2 


("9423428") .PN. 


USPAT; 


2003/11/19 


16 


14 








US-PGPUB; 














EPO; JPO; 














DERWENT; 














IBM TDB 








- 


23 


((real adj player$3) (media adj player$3) 


USPAT 


2003/11/20 


11 


42 






(quicktime$3) ) with (video near9 














recorder$5) 












1 


6353461. pn. and (start$3 end$6) 


USPAT 


2003/11/20 


17 


07 




1 


6353461. pn. and (time$3 with comment$5) 


USPAT 


2003/11/20 


16 


43 




1 


6353461. pn. and (start$3 begin$9 end$6) 


USPAT 


2003/11/20 


18 


10 




1 


6353461. pn. and (frame$4) 


USPAT 


2003/11/21 


14 


54 
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Figure 12: The ceedit display 

Figure 12 shows the main interface to ceedit. The tool displays the computed shot boundaries in three sections. The 
video is displayed in the upper left section with VCR controls that allow the video to be played at normal speed or 
advanced frame-by-frame. A horizontal timeline is displayed to the right of the video with shots marked by gray 
boundary lines and the current position marked as a vertical black line. The timeline allows a user to select and 
examine individual shots. A keyframe display is shown at the bottom of the interface that displays several keyframes 
from the neighborhood of the selected shot boundary. These three sections present an overview of the video in a form 
designed to allow the user to judge whether or not the computed boundary is accurate. 

There are several ways for a user to navigate through the computer-generated index of shots. A user can control the 
current position of the video stream, scanning through the video in order to quickly determine whether the computed 
shots are reasonably accurate. The control bar located below the video display provides the basic controls for manua 
navigating through the video. Alternatively, a user can select individual shots and observe the keyframes around the 
neighborhood of a shot boundary. The shot start and end positions are displayed for the current selection in the 
timeline display. These positions can be edited through the type-in fields or adjusted in frame increments via the 
arrows. The start and end positions can also be adjusted directly from the timeline by dragging the boundary marker 
The video control bar allows the user to position the video display at the start or end of the selected shot and move 
forward and backward frame-by-frame, to verify the accuracy of a computed shot boundary. 

A selected region of the video stream can be played to determine if it contains a shot boundary. After determining 
whether a boundary is accurate or not, it can be adjusted or deleted altogether. Similarly, new boundaries can be 
inserted if the algorithm misses a shot boundary. Two basic commands are provided to add or delete boundaries. The 
Insert Shot Boundary menu command adds a new shot boundary at the current position within the video stream. Th 
operation breaks the shot containing the current position into two separate shots. The Delete Shot Boundary menu 
command removes one of the boundaries of the currently selected shot. By default, the user is prompted to select wh 
boundary to delete. The delete operation effectively merges the two shots on either side of the deleted boundary. 

All of the actions above allow a user to correct mistakes in the computed shot boundaries. An algorithm may detect a 
transition which does not correspond to an actual shot boundary or it may fail to detect a shot boundary. In addition, 
computed shot boundary may be correct, but offset by several frames from the actual transition. In all of these cases, 
these inaccuracies should be corrected before the shot index information is added to the database. 

When a user is ready to insert the shot index into the database, ceedit queries the user for the document, segment an 
scene which will become the parents for the shot index. If the segment and scene indices are missing for the selected 
document, they can be automatically created. If the selected scene is already associated with a set of shots, the interf 
prompts the user as to whether the existing shots should be replace, or merged with the newly generated index. The 
merge operation compares the existing shot index with the index generated by ceedit. Two separate shot boundarie 
within a tolerance of a few frames of each other are merged into a single shot entry with the boundary marked at the 
midpoint. 

The content extraction tool interfaces introduce into the VOD system a suite of shot detection algorithms which can 
used to help a user annotate the shot boundaries of a document. Using the automated shot detection process, the 
complex task of identifying and marking shot boundaries is reduced to a simpler process of verification. 
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The content extraction shot boundary editor, named ceedit, provides an interface to examine the output of the shot 
detection algorithms and convert it to a format that can be stored in the database. The shot detection algorithms prod 
a file describing the frame numbers at which shot boundaries were detected, ceedit reads this file and displays a 
timeline, with shot boundaries marked as shown in figure 12. A user can then correct the shot boundaries before the 
information is added to the database. 
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